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Abstract 

The performance of maximum-likelihood (ML) decoded binary linear block codes is addressed 
' via the derivation of tightened upper bounds on their decoding error probability. The upper bounds 

on the block and bit error probabilities are valid for any memoryless, binary-input and output- 
symmetric communication channel, and their effectiveness is exemplified for various ensembles of 
C — > ' turbo-like codes over the AWGN channel. An expurgation of the distance spectrum of binary linear 

block codes further tightens the resulting upper bounds. 

o 

O 



1 Introduction 



Since the advent of information theory, the search for efficient coding systems has motivated 
the introduction of efficient bounding techniques tailored to specific codes or some carefully 
chosen ensembles of codes. The incentive for introducing and applying such bounds has 
strengthened with the introduction of various families of codes defined on graphs which 
closely approach the channel capacity with feasible complexity (e.g., turbo codes, repeat- 
accumulate codes, and low-density parity-check (LDPC) codes). Clearly, the desired bounds 
must not be subject to the union bound limitation, since for long blocks these ensembles of 
turbo-like codes perform reliably at rates which are considerably above the cutoff rate (Ro) 
of the channel (recalling that union bounds for long codes are not informative at the portion 
of the rate region above Rq, where the performance of these capacity-approaching codes is 
most appealing). Although maximum-likelihood (ML) decoding is in general prohibitively 
complex for long codes, the derivation of bounds on the ML decoding error probability is of 
interest, providing an ultimate indication of the system performance. Further, the structure 
of efficient codes is usually not available, necessitating efficient bounds on performance to 
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rely only on basic features, such as the distance spectrum and the input-output weight 
enumeration function (IOWEF) of the examined code (for the evaluation of the block and 
bit error probabilities, respectively, of a specific code or ensemble). 

A basic inequality which serves for the derivation of many previously reported upper 
bounds is the following: 



where y denotes the received vector at the output of the receiver, 1Z is an arbitrary geo- 
metrical region which can be interpreted as a subset of the observation space, and c is an 
arbitrary transmitted codeword. This category includes the Berlekamp tangential bound [I] 
where the volume 7Z is a half-space separated by a plane, the sphere bound by Herzberg 
and Poltyrev jTTj where 1Z is a hyper-sphere, Poltyrev's tangential-sphere bound ^1] (TSB) 
where 7Z is a circular cone, and Divsalar's bound [S] where 7Z is a hyper-sphere with an 
additional degree of freedom with respect to the location of its center. 

Another approach is the Gallager bounding technique which provides a conditional upper 
bound on the ML decoding error probability given an arbitrary transmitted (length-iV) 
codeword c m (P c \ m ). The conditional decoding error probability is upper bounded by 



where < p < 1 and A > (see |Hl HH] ; in order to make the presentation self-contained, 
it will be introduced shortly in the next section as part of the preliminary material). Here, 
^]v(y) i s an arbitrary probability tilting measure (which may depend on the transmitted 
codeword c m ), and Pjv(y|c) designates the transition probability measure of the channel. 
Connections between these two seemingly different bounding techniques in (0) and (j2J) were 
demonstrated in [21], showing that many previously reported bounds (or their Chernoff 
versions) whose derivation originally relied on the concept shown in inequality can in 
fact be re-produced as particular cases of the bounding technique used in (J2J). To this end, 
one simply needs to choose the suitable probability tilting measure ipN which serves as the 
"kernel" for reproducing various previously reported bounds. The observations in [21] relied 
on some fundamental results which were reported by Divsalar [Oj. 

The tangential-sphere bound (TSB) of Poltyrev often happens to be the tightest upper 
bound on the ML decoding error probability of block codes whose transmission takes place 
over a binary-input AWGN channel. However, in the random coding setting, it fails to 
reproduce the random coding exponent JU] while the second version of the Duman and 
Salehi (DS2) bound, to be reviewed in the next section, does (see [21])- The Shulman-Feder 
bound (SFB) can be derived as a particular case of the DS2 bound (see [21]), and it achieves 
the random coding error exponent. Though the SFB is informative for some structured linear 
block codes with good Hamming properties, it appears to be rather loose when considering 
sequences of linear block codes whose minimum distance grows sub-linearly with the block 
length, as is the case with most capacity-approaching ensembles of LDPC and turbo codes. 
However, the tightness of this bounding technique is significantly improved by combining the 
SFB with the union bound; this approach was exemplified for some structured ensembles of 
LDPC codes (see e.g., ^3] and the proof of fHl Theorem 2.2]). 



Pr(word error | c) < Pr(word error | c, y 



G K) + Pr(y i TZ) 



(1) 




(2) 
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In this paper, we introduce improved upper bounds on both the bit and block error 
probabilities. Section El presents some preliminary material. In Sectional we introduce an 
upper bound on the block error probability which is in general tighter than the SFB, and 
combine the resulting bound with the union bound. Similarly, an appropriate upper bound 
on the bit error probability is introduced. The effect of an expurgation of the distance 
spectrum on the tightness of the resulting bounds is considered in Section |U By applying 
the new bounds to ensembles of turbo-like codes over the binary-input AWGN channel, 
we demonstrate the usefulness of the new bounds in Section 03 especially for some coding 
structures of high rates. We conclude our discussion in Section |H1 For an extensive tutorial 
paper on performance bounds of linear codes, the reader is referred to [19J. 



2 Preliminaries 



We introduce in this section some preliminary material which serves as a preparatory step 
towards the presentation of the material in the following sections. 



2.1 The DS2 Bound 



The bounding technique of Duman and Salehi originates from the 1965 Gallager bound. 
Let ?p™(y) designate an arbitrary probability measure (which may also depend on the trans- 
mitted codeword x m ). The Gallager bound [TU] can then be put in the form (see |2T] ) 



y \ m'^m 

By invoking the Jensen inequality in (jSJ) for < p < 1, the DS2 bound results 

P elm < ( J2 E^(yl C ™)" ^(y) 1 -" ( PN ^ m,) ) X ) , < p < 1, A > 0. (4) 

Wn y \PN(y\Cm)J J 

Let G^(y) be an arbitrary non-negative function of y, and let the probability density function 
«y)be 

M) -J2 y G^y)p N (y\c m ) (5) 

The functions G™(y) and if)™(y) are referred to as the un-normalized and normalized tilting 
measures, respectively. The substitution of (JHJ) into (J1J) yields the following upper bound on 
the conditional ML decoding error probability 

P«| m < fee^(y) p N (y\c m )\ 

E 5>(yM os(y)-i (^fey)T- M6) 

ym'^m y x ' / 
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The upper bound (JHJ) was also derived in [HI Eq. (62)]. 



N 

For the case of memoryless channels, and for the choice of ip^iy) as ip™(y) = Y\ ^{v-i) 

i=i 

(recalling that the function ip™ may depend on the transmitted codeword x m ), the upper 
bound (jlj is relatively easily evaluated (similarly to the standard union bounds) for linear 
block codes. In that case, (JH) is calculable in terms of the distance spectrum of the code, 
not requiring the fine details of the code structure. Moreover, (j3J) is also amenable to some 
generalizations, such as for the class of discrete memoryless channels with arbitrary input 
and output alphabets. 



2.2 The Shulman and Feder bound 



We consider here the transmission of a binary linear block code C where the communica- 
tion takes place over a memoryless binary-input output-symmetric (MBIOS) channel. The 
analysis refers to the decoding error probability under soft-decision ML decoding. 

The Shulman and Feder bound (SFB) [20] on the block error probability of an (N, K) 
binary linear block code C, transmitted over a memoryless channel is given by 

P e < ( 7 ) 

where 



EAR) = max (E (p) - pR) 

0<p<l 



(9) 



ce(C) = ^rnax - — — p ^ N . (10) 



E T is the random coding error exponent |10j . R — j- designates the code rate in bits per 
channel use, and 

A 

1<1<N 2~N(l-R) ( N \ 

In the RHS of ()10j). {A{\ denotes the distance spectrum of the code. Hence, for fully random 
block codes, a(C) is equal to 1, and the Shulman- Feder bound (SFB) particularizes to the 
random coding bound ^Oj- In general, the parameter a(C) in the SFB (0) measures the 
maximal ratio of the distance spectrum of a code (or ensemble) and the average distance 
spectrum which corresponds to fully random block codes of the same block length and rate. 

The original proof of the SFB is quite involved. In |21j . a simpler proof of the SFB is 
derived, and by doing so, the simplified proof reproduces the SFB as a particular case of the 
DS2 bound (see Eq. (jlj)). In light of the significance of the proof concept to the continuation 
of our paper, we outline this proof briefly. 

Since we deal with linear block codes and the communication channel is memoryless, 
binary- input output-symmetric channel (MBIOS), one can assume without any loss of gen- 
erality that the all zero codeword c is the transmitted vector. In order to facilitate the 
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expression of the upper bound (jBJ) in terms of distance spectrum of the block code C, we 
consider here the case where the un-normalized tilting measure G° N (y) can be expressed in 
the following product form: 



N 



GSr(y) = II^ 



(11) 



1=1 



where g is an arbitrary non-negative scalar function, and the channel is by assumption 
MBIOS, so that the transition probability measure is expanded in the product form 



N 



P;v(y|c m >) = JJp(?/i|c mV ) 



(12) 



i=i 



where c m / = (tVn'.ij • • • > Cm'.jv)- Hence, the upper bound on the conditional ML decoding 
error probability given in @ can be rewritten as 



Pe — Pc\0 



N(l-p) 



< 



^2g(y) p(y|o) 

y / 



N-l 



n p 



i=i 



< max 



A > 0, 
< p < 1 



^2g(y) p(y\o) 



N(l-p) 



o<i<n 2-N(i-R)(*\ 

Y,^yf' lp p{y\^) + Y,9(yf~ lp p{yW' x pW 



2 -N(l-R)p 
Np 



By setting 



g{y) 



p(y\Q) i+p, A 



1 + p 



(13) 



(14) 



and using the symmetry of the channel (where p(y\0) = p(—y\l)), the SFB follows readily. 



2.3 The Tangential- Sphere Bound 

The tangential-sphere bound (TSB) on the block error probability of ML decoding was 
derived by Poltyrev (see ^21 E]), and it applies to the case where the transmission takes 
place over an AWGN channel, and the modulated signals have constant energy. 

Consider a binary linear block code C with length N, and suppose that the modulated 
codewords are transmitted over a binary-input AWGN channel, using an equi-energy mod- 
ulation (i.e., all the transmitted signals have the same energy, E s , and can be therefore 
represented as points on an iV-dimensional sphere whose center is located at the origin). 
The TSB is based on inequality (0), where the chosen volume 1Z is an iV-dimensional cir- 
cular cone, whose vertex is located at the origin and whose central line passes through the 
origin and point which corresponds to the transmitted signal (see Fig. EJ. This circular cone 
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has a half angle of 9 and a radius r, and the optimization is carried over r (r and 6 are 
related, as shown in Fig. QJ). Let us designate the circular cone in Fig. HJby Cn(9). 

Let z = (zi, z 2 , ■ ■ ■ , z N ) designate an n-dimensional noise vector which corresponds to iV 
orthogonal projections of the AWGN. Let z\ be the radial component of z (see Fig.GJ, so the 
other N — 1 components of z are orthogonal to its radial component. Since z is a Gaussian 
vector and its components are un-correlated, then the N components of z are i.i.d., and each 
component has a zero mean and variance a 2 = 





Figure 1: The geometric interpretation of the TSB. 



Let y = s + z be the received vector at the output of the binary-input AWGN chan- 
nel. Given the value of the radial component of the noise vector, z\, the conditional error 
probability satisfies 

Prob (E( Zl )\ zi) < Prob (E( Zl ), y G C N (6) | z x ) + Prob (y i C N (6) | Zi) (15) 
and from the union bound 

Prob (E(zx), y G C N (6) \ z x ) < J] A k Prob (E k ( Zl ), y G C N (6) \ Zl ) (16) 

k 
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where Ak designates the number of the constant-energy signals (sj) in the considered signal 
set so that their Euclidean distance from the transmitted signal (so) is 5k- We note that 
for BPSK modulated signals where s = (2c — l)y/E~l, the Euclidean distance between the 
two signals Sj and s is directly linked to the Hamming weight of the codeword Cj. Let the 
Hamming distance between the two codewords be k, i.e., wn(ci) = k, then the Euclidean 
distance between the two BPSK modulated signals is equal to 5k = 2y/kE s . In the latter 
case, Ak is the number of codewords of the code C with Hamming weight k (i.e., {Ak} is the 
distance spectrum of the linear code C). 



The combination of Eqs. (jTojl and (jlojl gives 

Prob (E( Zl )\ z x ) < J2{ Ak Prob y G Cn ^ I 

k 

+Prob (y £ CW(0) \ Zl ). (17) 
From Fig. ^ we obtain the following equalities: 



fNE B 
5fe 



"2r" 



1-- 



and let 



ANE, 



The TSB on the block error probability under ML decoding depends on the distance 
spectrum {Ai}^L Q of the block code C, and by invoking the smoothing theorem w.r.t. zx, the 
bound finally admits the following form: 



P*< I 4^e~^< fc*** ' ' < (19) 



-x V27RT 1 - _ / _ l r 2 



+ 1-7 



2 ' 2a 2 



where 

^( a ,x) = ^ T I t a ~ l e~ l dt, a,x>0 (20) 
T(a) J 

denotes the normalized incomplete Gamma function. In order to get the tightest bound 
within the form one needs to optimize the radius r. This optimization is carried 

by setting to zero the derivative of the RHS of (fTTJj) w.r.t. r, which yields the following 
optimization equations ([Ej): 



<«fc 



cos" 1 ' & 




•JV-2> 



(21) 
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In |15| Appendix B], Sason and Shamai prove the existence and uniqueness of the solution to 
the optimization equation (|21j) . and introduce a simple algorithm for solving this equation 
numerically. In order to derive an upper bound on the bit error probability, let A w ^ designate 
the corresponding coefficient in the IOWEF which is number of codewords which are encoded 
by information bits whose number of ones is equal to w (where < w < NR) and whose 
Hamming weights (after encoding) are equal to k, and define 

NR. 

A * = £ (m) ^ k = >---> N - ( 22 ) 

w=l 

In [H3 Appendix C], Sason and Shamai derive an upper bound on the bit error probability 
by replacing the distance spectrum {Ak} in (JTJH) and (jUj) with the sequence {A' k }, and show 
some properties on the resulting bound on the bit error probability. 

3 Improved Upper Bounds 

3.1 Upper Bound on the Block Error Probability 

It is well known that at rates below the channel capacity, the block error probability of the 
ensemble of fully random block codes vanishes exponentially with the block length. In the 
random coding setting, the TSB fails to reproduce the random coding exponent, while 
the SFB [20] particularizes to the 1965 Gallager bound for random codes, and hence, the SFB 
reproduces the random coding exponent. The SFB is therefore advantageous over the TSB 
in the random coding setting when we let the block length be sufficiently large. Equations 
(J7J) and (fTUj) imply that for specific linear codes (or ensembles), the tightness of the SFB 
depends on the maximal ratio between the distance spectrum of the code (or the average 
distance spectrum of the ensemble) and the average distance spectrum of fully random block 
codes of the same length and rate which has a binomial distribution. 

In order to tighten the SFB bound for linear block codes, Miller and Burshtein [T3*] 
suggested to partition the original linear code C into two subcodes, namely C and C"; the 
subcode C contains the all-zero codeword and all the codewords with Hamming weights of 
I G tl C {1,2, ...,N}, while C" contains the other codewords which have Hamming weights 
of I G U c = {1, 2, N} \ U and the all-zero codeword. From the symmetry of the channel, 
the union bound provides the following upper bound on the ML decoding error probability: 

P e = P el o<P e \a(C') + P el o(C") (23) 

where P c |o(C) and P e |o(C") designate the conditional ML decoding error probabilities of C 
and C", respectively, given that the all zero codeword is transmitted. We note that although 
the code C is linear, its two subcodes C and C" are in general non-linear. One can rely on 
different upper bounds on the conditional error probabilities P e \ (C) and P e | (C"), i.e., we 
may bound P c \q(C) by the SFB, and rely on an alternative approach to obtain an upper bound 
on P C | (C"). For example, if we consider the binary-input AWGN channel, then the TSB (or 
even union bounds) may be used in order to obtain an upper bound on the conditional error 
probability P e \ (C") which corresponds to the subcode C" . In order to obtain the tightest 
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bound in this approach, one should look for an optimal partitioning of the original code C 
into two sub-codes, based on the distance spectrum of C. The solution of the problem is quite 
tedious, because in general, if the subset U can be an arbitrary subset of the set of integers 
{1, . . . , N}, then one has to compare J2iLo (T) = ^ N different possibilities for U. However, 
we may use practical optimization schemes to obtain good results which may improve the 
tightness of both the SFB and TSB. 

An easy way to make an efficient partitioning of a linear block code C is to compare its 
distance spectrum (or the average distance spectrum for an ensemble of linear codes) with 
the average distance spectrum of the ensemble of fully random block codes of the same rate 
and block length. Let us designate the average distance spectrum of the ensemble of fully 
random block codes of length N and rate R by 

B l A 2-^(1-*) l = 0,l,...,N. (24) 

Then, it is suggested to partition C in a way so that all the codewords with Hamming weight 
I for which ^ is greater than some threshold (which should be larger than 1 but close to 
it) are associated with C", and the other codewords are associated with C. The following 
algorithm is suggested for the calculation of the upper bound on the block error probability 
under ML decoding: 

Algorithm 1 

1. Set 

U = <$>, U c = {1,2, ...A}, 1 = 1 
where $ designates an empty set, and set the initial value of the upper bound to be 1. 

2. Compute the ratio ^ where {^4;} is the distance spectrum of the binary linear block code 
(or the average distance of an ensemble of such codes), and {B{\ is the binomial distribution 
introduced in (J23J. 

3. If this ratio is smaller than some threshold (where the value of the threshold is typically 
set to be slightly larger than 1), then the element I is added to the set U, i.e., 

U:=U + {1}, U C :=U C \{1}. 

4. Update correspondingly the upper bound in the RHS of (J2*H|) (we will derive later the 
appropriate upper bounds on P e \ Q {C') and P e \ (C"). 

5. Set the bound to be the minimum between the RHS from Step 4 and its previous value. 

6. Set 1 = 1 + 1 and go to Step 2. 

7. The algorithm terminates when I gets the value A (i.e., the block length of the code) or 
actually, the maximal value of / for which Ai does not vanish. 1 

lr The number of steps can be reduced by factor of 2 for binary linear codes which contain the all-ones 
codeword (hence maintain the property A\ = Ajy_ For such codes, the update equation in Step 3 becomes: 
U := U + {I, N — I}, U c := U c — {l,N — 1} and the algorithm terminates when I gets the value . 
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Fig. |2^a) shows a plot of the ratio ^ as a function of 5 = for an ensemble of uniformly 
interleaved turbo-random codes. The calculation of the average distance spectrum of these 
ensemble relies on the results of Soljanin and Urbanke in [22! ■ F rom the discussion above, it is 
clear that the combination of the SFB with another upper bound has the potential to tighten 
the overall upper bound on the ML decoding probability; this improvement is expected 
to be especially pronounced for ensembles whose average distance spectrum resembles the 
binomial distribution of fully random block codes over a relatively large range of Hamming 
weights, but deviates significantly from the binomial distribution for relatively low and large 
Hamming weights (e.g., ensembles of uniformly interleaved turbo codes possess this property, 
as indicated in |T3J Section 4]). This bounding technique was successfully applied by Miller 
and Burshtein [T3] and also by Sason and Urbanke [TH] to ensembles of regular LDPC codes 
where the SFB was combined with union bounds. If the range of Hamming weights where the 
average distance spectrum of an ensemble resembles the binomial distribution is relatively 
large, then according to the above algorithm, one would expect that C typically contains 
a very large fraction of the overall number of the codewords of a code from this ensemble. 
Hence, in order to obtain an upper bound on P C | (C"), where C" is expected to contain a 
rather small fraction of the codewords in C, we may use a simple bound such as the union 
bound while expecting not to pay a significant penalty in the tightness of the overall bound 
on the decoding error probability (P e ). 

The following bound introduced in Theorem ^ is derived as a particular case of the DS2 
bound [Hj. The beginning of its derivation is similar to the steps in [211 Section 4A], but we 
later deviate from the analysis there in order to modify the SFB. We finally obtain a tighter 
version of this bound. 

Theorem 1 (Modified Shulman and Feder Bound). Let C be a binary linear block 
code of length N and rate R, and let {^} designate its distance spectrum. Let this code be 
partitioned into two subcodes, C and C", where C contains the all-zero codeword and all the 
other codewords of C whose Hamming weights are in an arbitrary set U C {1, 2, , . . . , iV}; the 
second subcode C" contains the all- zero codeword and the other codewords of C which are not 
included in C . Assume that the communication takes place over a memoryless binary-input 
output-symmetric (MBIOS) channel with transition probability measure p(y\x), x e {0, 1}. 
Then, the block error probability of C under ML decoding is upper bounded by 



where 

P,|o(C) < SFB(p) 



£ 

.leu 



N 



P e <P e \ (C')+P elQ (C") 

A( P ) \7 B(p) 



I J \A(p)+B(p)J \A(p)+B(p) 



N-n p 



< p < 1 (25) 



^(p) = £my|0My|i)Fk 



£(P)=£^P(V|0) 



2 

1+p 



p-1 



~p(y\0)i+» + ^p{y\l)'+» 



(26) 
(27) 



The multiplicative term, SFB(p), in the RHS of (|25|) designates the conditional Shulman- 
Feder upper bound of the subcode C given the transmission of the all- zero codeword, i.e., 
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SFB(p) = i o < p < 1 (28) 

and Eq is introduced in An upper bound on the conditional block error probability for 
the subcode C", P e \ (C"), can be either a standard union bound or any other bound. 

Proof. Since the block code C is linear and the channel is MBIOS, the conditional block 
error probability of C is independent of the transmitted codeword. Hence, the union bound 
gives the following upper bound on the block error probability: P e < P e \ (C') + P e \ (C"). 

In order to prove the theorem, we derive an upper bound on P e | (C'). Let {Ai(C')} denote 
the weight spectrum of the subcode C, and let Gjv(y) be an arbitrary non-negative function 
of the received vector y = (yx, y 2 , . . . , 2/at) where this function is assumed to be expressible 
in the product form (fTTj) . Then, we get from (JHJ) and the following upper bound on the 
conditional ML decoding error probability of the subcode C: 

^ N{l-p) 



V 

N-l 



^ff(y)H P (y|o)J ( Xy^Kylo) 1 -^!! 

v N(l-p) 




E ( (AT) ) (7)1 T.tiv^Mo) 



leu 




N-l 



< 





Y^g(y)p(ym 2- N ^ 



N-l 



Y,9{yf'"pm [Y,9{y) x MvW- x pm x ) ) ■ w 




The transition in the first equality above follows since Ai(C) = for I ^ U, and Ai(C) 
coincide with the distance spectrum of the code C for all I G U. Note that (j2HJ) is a tighter 
version of the bound in [2H Eq. (32)]. The difference between the modified and the original 
bounds is that in the former, we only sum over the indices I G U while in the latter, we sum 
over the whole set of indices, i.e., I G {1,2,..., iV}. By setting the tilting measure in ()14j) . 
the symmetry of the MBIOS channel gives the equality 



1 i 1 i 

-p(y\0)—p + -p(y\l)^ 



(30) 
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and from (j2o|) and (jzj 

^KylO) 1 -^!)^) 1 -' 



eI 



[p(y|0)p(y|l)]i+p 



-p{y\0)~p +-p(y|l)^ 



£<p(y|o) 



2 

1+P 



-p(j/|0)^ + -p(?/|l)^ 



p-1 



(31) 



(32) 



where the RHS of (|3~TJl and (J32j) are obtained by setting A = Finally, based on (JUjl and 
the symmetry of the channel, one can verify that 



(33) 



Substituting (|3l] j) - (j3^j) into gives the following conditional upper bound on the ML 
decoding error probability of the subcode C: 



P e \o(C) < a(C'Y 



A{p) + B{p) 



N(l-p) 



Jeu 



N 



2 -*(i-4>p. £ ]\A\p)B»-\p) 



(34) 



where we use the notation 



a(C) = max ■ 



A, 



leu 2- N ( 1 - R )( p })' 

The latter parameter measures by how much the (expected) number of codewords in the 
subcode C deviates from the binomial distribution which characterizes the average distance 
spectrum of the ensemble of fully random block codes of length N and rate R. By straight- 
forward algebra, we obtain that 



P e \o(C) < a(C) 

£ 

jeu 

a(cy 

SFB(p) 



A(p) + B(p) 



TV 



2 -N(l-R)p I 1 

2 



-Np 



N 



A(p) 



I J \A{p)+B{p)J \A(p)+B(p) 



B{ P ) 



N-l 



A(p) + B{p) 



.leu v 



TV 



2 NRp 

A(p) 



E 

leu 



N 



A(p) 



B{ P ) 



I J \A(p) + B(p)J \A(p) + B(p) 

N-l 



N-l 



A(p) + B(p)J \A(p) + B(p) 



B(P) 



, < p < 1. (35) 



12 



The second equality follows from (}28|) and ©, and since 



E (P) = -log 2 \J2 



1 j 1 i 1 1+p 

-p(y\0)^ + -p(y\l)^ 



log2 (m+m) . (36) 



2 

This concludes the proof of the theorem. □ 

Discussion: The improvement of the bound introduced in Theorem over the standard 
combination of the SFB and the union bound ^Sl EE] stems from the introduction of the 
factor which multiplies SFB(p) in the RHS of (|2"K|) : this multiplicative term cannot exceed 1 
since 



E 



N\f A(p) V/ B(p) ^ 



^ J J \A(p) + B(p)J \A(p) + B(p) 

< y ( N ) ( m v ( m y~ l = i 

-2^ \ i J \A(p) + B(p)J \A(p) + B(p)l 



1=0 



This multiplicative factor which appears in the new bound is useful for finite-length codes 
with small to moderate block lengths. The upper bound (|23j) on P C | (C) is clearly at least as 
tight as the corresponding conditional SFB. We refer to the upper bound (|23j) as the modified 
SFB (MSFB). The conditional block error probability of the subcode C" , given that the all- 
zero codeword is transmitted, can be bounded by a union bound or any improved upper 
bound conditioned on the transmission of the all-zero codeword (note that the subcode 
C" is in general a non- linear code). In general, one is looking for an appropriate balance 
between the two upper bounds on P$ and P$ (see Algorithm 1). The improvement that 
is achieved by using the MSFB instead of the corresponding SFB is exemplified in Section El 
for ensembles of uniformly interleaved turbo-Hamming codes. 



3.2 Upper Bounds on Bit Error Probability 

Let C be a binary linear block code whose transmission takes place over an arbitrary MBIOS 
channel, and let P^ designate the bit error probability of C under ML decoding. In ^f)J 
Appendix A], Sason and Shamai derived an upper bound on the bit error probability of 
systematic, binary linear block codes which are transmitted over fully interleaved fading 
channels with perfect channel state information at the receiver. Here we generalize the 
result of for arbitrary MBIOS channels. In order to derive the desired upper bound we 
use the following lemma due to Divsalar [6 , and provide a simplified proof to this lemma: 

Lemma 1. [6, Section III.C] Let C be a binary block code of dimension K whose trans- 
mission takes place over an MBIOS channel. Let C(w) designate a sub-code of C which 
includes the all-zero codeword and all the codewords of C which are encoded by information 
bits whose Hamming weight is w. Then the conditional bit error probability of C under ML 
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decoding, given that the all-zero codeword is transmitted, is upper bounded by 



P b |o < $>jv(y|o; 



l-Xp 



E (I) E MyV 

W=l c 6 C(ui) 

c 



A > 0, < p < 1. (37) 



We introduce here a somewhat simpler proof than in 



Proof. The conditional bit error probability under ML decoding admits the form 

'wo(y)' 



ft, - E (^) 



P7v(y|0) 



(38) 



where u>o(y) G {0, 1,...,K} designates the weight of the information bits in the decoded 
codeword, given the all-zero codeword is transmitted and the received vector is y. In par- 
ticular, if the received vector y is included in the decision region of the all-zero codeword, 
then wo(y) = 0. The following inequalities hold: 




A > 



(39) 



Inequality (a) holds since the received vector y falls in the decision region of a codeword c 
which is encoded by information bits of total Hamming weight Wo(y); hence, the quotient 
p^jyjo) i s larger than 1 while the other terms in the sum are simply non-negative. The third 
inequality holds because of adding non-negative terms to the sum. The lemma follows by 
substituting flU into the RHS of IjBBjl . □ 

Theorem 2. (The SFB Version on the BER) Let C be a binary linear block code of 
length N and dimension K, and assume that the transmission of the code takes place over 
an MBIOS channel. Let A w j designate the number of codewords in C which are encoded by 
information bits whose Hamming weight is w and their Hamming weight after encoding is 
/. Then, the bit error probability of C under ML decoding is upper bounded by 



(40) 



where R = % is the code rate of C, and 



N 



Ob(C) 



max 



4 



o<z<iv 2~ N ( 1 - R ) ( N ) 



w=l 



14 



Proof. Due to the linearity of the code C and the symmetry of the channel, the conditional 
bit error probability of the code is independent on the transmitted codeword; hence, without 
any loss of generality, it is assumed that the all- zero codeword is transmitted. From (|37J) . 
the following upper bound on the bit error probability of C follows: 

p b = p b |o < E^(yi°) 1_Ap |E(f) E ^(yi c ) A [ > a>o, o< P <i 



w=l 



c e C(w) 

c / O 



K 



E^(y)^Sr(y)-^(y|o)"E(f) E 



W = l 



c e C(wy 

c ^ O 



pn (y\c) 
pn (y|0) 



1 A 



(41) 



where ip% is an arbitrary probability tilting measure. By invoking Jensen inequality in 
the RHS of (|41jl and replacing ijj%(y) with the un-normalized tilting measure G° N (y) which 
appears in the RHS of (0), the upper bound in (j4~Tj) transforms to 



E (i) E Z»(»l«y)^ 



c e c(»)y 

c / O 



pjy(y|c) 
pjv(y|°) 



, < p < 1, A > 0.(42) 



We consider an un-normalized tilting measure G° N (y) which is expressible in the product 
form (fTTJ) . Since the communication channel is MBIOS and C is a binary linear block code, 
one obtains the following upper bound on the bit error probability: 



Pb|o < 



< 



N(l-p) 



< p < 1, A>0 



E^EmE^^m 1 ^ 

V 

E^) p(y|o) 



5^P(2/|1) A P(2/|0) 1 - A ^( 2 /) 1 -^ 



1=0 \ y 

JV(l-p) 



AT-Z 



E^ E^i )^) 1 ^ E^i 1 )^ ) 1 ^) 1 ^ 



1=0 



^2g{y) p(y\o) 



N(l-p) 



A' 



2 -JV(l-Jl)p 



Np 



?/ y / 

By setting g(y) as in (jl4j) . we obtain an upper bound which is the same as the original 
SFB, except that the distance spectrum {Ai} is replaced by {^4/}. This provides the bound 
introduced in (|4T)|) . and concludes the proof of the theorem. □ 
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Similarly to the derivation of the combined upper bound on the block error probability 
in Theorem we suggest to partition the code into two subcodes in order to get improved 
upper bounds on the bit error probability; however, since we consider the bit error probability 
instead of block error probability, the threshold in Algorithm 1 is typically modified to a value 
which is slightly above | (instead of 1). Since the code is linear and the channel is MBIOS, 
the conditional decoding error probability is independent of the transmitted codeword (so, 
we assume again that the all- zero codeword is transmitted). By the union bound 

Pb = Pb|o < Pb\o(C) + P b |o(C") (44) 

where Pb\o(C) and Pb\o(C") denote the conditional ML decoding bit error probabilities of two 
disjoint subcodes C and C" which partition the block code C (except that these two subcodes 
have the all-zero vector in common), given that the all- zero codeword is transmitted. The 
construction of the subcodes C and C" is characterized later. 

Upper bound on P^\ (C): Let A w j designate the number of codewords of Hamming weight 
I which are encoded by a sequence of information bits of Hamming weight w. Similarly to 
the discussion on the block error probability, we use the bit-error version of the SFB (see 
Eq. ASH) ) as an upper bound on P b | (C). From Theorem it follows that the conditional bit 
error probability of the subcode C, given that the all- zero codeword is transmitted is upper 
bounded by 

-NE I (R+ losab(C ' ) 



Pb|o(C')<2 V - ; (45) 
where 

4(0 ^AlE^te)^, if Jew 



a h (C) = max , ^ <^ ^- =1 ^r) ~W ^ « ( 46 ) 

v ; leu Bi ' v ; \ otherwise v 1 

and the set U in stands for an arbitrary subset of {1, ... , N}. 

Upper bound on Pb\o{C"): We may bound the conditional bit error probability of the sub- 
code C" , Pb\o(C"), by an improved upper bound. For the binary-input AWGN, the modified 
version of the TSB, as shown in ^1 is an appropriate bound. This bound is the same as 
the original TSB in (fTTIj) . except that the distance spectrum {Ai} is replaced by {A/(C")} 
where 

{NR 
T,{m) A ^ {UeW (47) 
w=l 
otherwise 

and IA C stands for an complementary set of IA in gSJ, i.e., W = {1,...,N}\U. For the 
binary-input AWGN channel, the TSB on the conditional bit error probability admits the 
following final form (see [TH]): 

p b|o(r) < r -^e-A ( ^ { A1(c " ] ^) **} 



where the incomplete Gamma function 7 is introduced in ()20j) . As the simplest alternative to 
obtain an upper bound on the conditional bit error probability of the subcode C given that 
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the all-zero codeword is transmitted, one may use the union bound (UB) for the binary-input 
AWGN channel 



NR 



w=i ieu c 




(49) 



where E\> is the energy per information bit and 4^ is the two-sided spectral power density 
of the additive noise. 

In order to tighten the upper bound (|45jl . we obtain the bit-error version of the MSFB 
(see Eq. (125)1 ). by following the steps of the proof of Theorem [0 In a similar manner to the 
transition from (JJJ) to ([4*Uj) . we just need to replace the terms Ai(C) in (|2l)|) with A'^C) to get 
the conditional modified SFB (MSFB) on the bit error probability of C, given the all-zero 
codeword is transmitted. The resulting upper bound is expressed in the following theorem: 

Theorem 3 (Modified SFB on the Bit Error Probability). Let C be a binary linear 
block code of length N and rate R, and let A w j be the number of codewords of C which 
are encoded by information bits whose Hamming weight is w and their Hamming weight 
after encoding is I (where < w < NR and < I < N). Let the code C be partitioned 
into two subcodes, C and C", where C contains all codewords of C with Hamming weight 
I G U C {1, 2, , . . . , N} and the all- zero codeword, and C" contains the all-zero codeword 
and all the other codewords of C which are not in C. Assume that the communication takes 
place over an MBIOS channel. Then, the bit error probability of C under ML decoding is 
upper bounded by 

p b <Pb|o(e')+^b|o(e") 

where 



Puo(C) < 2 



-^( So (p)-p(R + ^#^)) 




A( P ) 



a h (C 



■l\ A 



leu 



A' 



A(p) + B(p)J \A(p) + B(p) 



B(p) 



N-ll p 



NR 



< p < 1 
(50) 



max 



N\ ' 



w=l 



w.l 



(|2T|) and 0, respectively. An upper 



and the functions A,B,E are introduced in 
bound on the conditional bit error probability for the subcode C", Pb\ (C"), can be either a 
union bound (j49)l . the TSB (|48|l or any other improved bound. 



Discussion: Note that «b (C) < ot(C), therefore the bound on the bit error probability 
in (jBTij) is always smaller than the bound on the block error probability in (J2SJ), as one could 
expect. 

In the derivation of the MSFB on the conditional block and bit error probabilities (see 
Eqs. (|23|) and (jHUj) . respectively), we obtain simplified expressions by taking out the max- 
imum of | A 'b { ^ | and j Al j^ - 1 from the corresponding summations in (|2*n|) and fj4Hjl . This 
simplification was also done in [21] for the derivation of the SFB as a particular case of the 
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DS2 bound. When considering the case of an upper bound on the block error probability, this 

simplification is reasonable because we consider the terms j - j which vary slowly over a 

large range of the Hamming weights I (see Fig.|2^a) when referring to ensembles of turbo-like 
codes whose average distance spectrum resembles the binomial distribution). However, by 

considering the terms < ^ ^ whose values change considerably with / (see Fig. Efb)), such 

simplification previously done for the block error analysis (i.e., taking out the maximal value 

A'(C') 

of g- from the summation) is expected to significantly reduce the tightness of the bound 
on the bit error probability. Thus, the modification which results in (J5Uj) does not seem in 
general to yield a good upper bound. 2 In order to get a tighter upper bound on the bit error 
probability we introduce the following theorem: 

Theorem 4 (Simplified DS2 Bound). Let C be a binary linear block code of length N and 
rate R, and let A w j designate the number of codewords which are encoded by information 
bits whose Hamming weight is w and their Hamming weight after encoding is I (where 
< w < NR and < I < N). Let the code C be partitioned into two subcodes, C and 
C", where C contains all the codewords in C with Hamming weight ! E W C {1,2,,..., N} 
and the all-zero codeword, and C" contains all the other codewords of C and the all-zero 
codeword. Let 



A/(C') A \ (^k) Aw ' 1 l{leU 

Al[L) ~\0 otherwi 



otherwise 

Assume that the communication takes place over an MBIOS channel. Then, under ML 
decoding, the bit error probability of C, is upper bounded by 

P b < Pmo(C) + P h \ (C") 

where 

a,„(c<) < 2 -H-( R+ ^)), <p<i (5i) 



A(p), B(p) and E are defined in (}26|) . (|2*7j) and ©, respectively. As before, an upper bound 
on the conditional bit error probability for the subcode C" , Ph\o{C"), can be either a union 
bound or any other improved bound. 



Proof. Starting from the first equality in (jlSJ), and using the definition for A(p), B(p) in 

2 Note that for an ensemble of fully random block codes, all the terms ^ are equal to |; hence, the 
simplification above does not reduce the tightness of the bound at all when considering this ensemble. 
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(JUJ) and (J2U we get 




\^ A V; \A(p) + B(p)J \A{p) + B{p))] 

(53) 

where 

5^2-^)^, l = 0,...,N 

designates the distance spectrum of fully random block codes of length N and rate R. Using 
the definition for a p (C) in (p)2"j) we get the upper bound (Jo"Tj) . □ 

Evidently, the upper bound is tighter than the bit-error version of the SFB in (j45)) . 

A' (C') 

because dt p {C) which is the expected value of ^ is not larger than a^yC ) which is the 

maximal value of ^ . We note that the upper bound (|5T|) is just the DS2 bound |H], with 
the un-normalized tilting measure (|14|) . This tilting measure is optimal only for the ensemble 
of fully random block codes, and is sub-optimal for other codes. We refer to the upper bound 
1)51 J) as the simplified DS2. From the discussion above, we conclude that the simplified DS2 
bound (which is also valid as an upper bound on the conditional block error probability if 
we replace A! t {C) in ()53|) by Ai(C')) is advantageous over the MSFB when A\ (or A[ for 
the case of block error probability) changes considerably over the Hamming weight range 
of interest. This is demonstrates for the block error probability of the ensemble of multiple 
turbo-Hamming codes where there is no noticeable improvement if we use the simplified DS2 
to bound P e |o(C) instead of the MSFB, where for the case of bit-error probability we get 
tighter upper bound when using the simplified DS2 to upper bound Pb|o(0 rather than the 
MSFB. 



4 Expurgation 

In this section we consider a possible expurgation of the distance spectrum which yields 
in general tighter upper bounds on the ML decoding error probability when transmission 
takes place over a binary-input AWGN (BIAWGN) channel. To this end, we rely on some 
properties of the Voronoi regions of binary linear block codes, as presented in PJ EJ IS] ■ 

Let C be a binary linear block code of length N and rate R. Without any loss of generality, 
let us assume that the all- zero codeword, Co, was transmitted over the BIAWGN channel. 
For any received vector y, an ML decoder checks whether it falls within the decision region 
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of the all zero vector. This decision region (which is also called the Voronoi region of Co) is 
defined as the set Vo of vectors in M. N that are closest (in terms of Euclidian distance) to the 
all- zero codeword, i.e., 

V ={xGl N : d(x, c ) < d(x, c), VceC}. (54) 

Not all of the 2^ inequalities in (|54|) are necessarily required to define the Voronoi region. 
The minimal set of codewords that determine the Voronoi region of c , forms the set of 
Voronoi neighbors of c (to be designated by Ao). So the region (|H4*jl can be defined by 

V = {x G R N : d(x,c ) < d(x,c), Vcg7V }. (55) 

It is clear that the block error probability of C is equal to the conditional block error proba- 
bility of the expurgated subcode C ex , assuming the all-zero codeword is transmitted, where 
C ex designates the subcode of C which contains the all-zero codeword and all its (Voronoi) 
neighbors. Hence, any upper bound that solely depends on the code distance spectrum of 
the code can be tightened by replacing the original distance spectrum with the distance spec- 
trum of the expurgated code. It should be noted, however, that the argument above cannot 
be applied to the bit error probability. This stems from the fact that while the block error 
event is solely defined by the Voronoi region of the transmitted codeword, the bit error event 
also depends on the Hamming weight of the information bits of each decoded codeword; 
hence, the above expurgation cannot be applied to the analysis of the bit error probability. 
The distance spectrum of the Voronoi neighbors of an arbitrary codeword of some popular 
linear block codes (e.g., Hamming, BCH and Golay codes) is given in pQ. A simple way to 
find a subcode of C which contains the subcode C ex is given in the following theorem from 

m 

Theorem 5 (On the Voronoi Regions of Binary Linear Block Codes For any 

binary linear block code C with rate R and length N 

Af ^{ceC:l< W n (c) < 2d min - 1} 

and 

No C {c e C : 1 < W n {c) < N(l -R) + l} 
where d m i n is the minimal Hamming weight of the codewords in C. 

Note that according to the theorem above, one should expect the expurgation to have 
maximal impact on the tightness of an upper bound for high rate codes, where most of the 
codewords can be expurgated. We should also observe that the expurgated codewords have 
large distances from the all-zero codeword (all the expurgated codewords have a Hamming 
weight larger than 2d min — 1). Thus, the improvement due to the expurgation process is 
especially substantial at low SNRs. One can use this theorem to achieve an immediate im- 
provement of an arbitrary upper bound by expurgating all the codewords whose Hamming 
weight is greater than N(l — R) + 1. We refer to this kind of expurgation as the trivial expur- 
gation. The trivial expurgation, though very simple to apply, does not produce satisfactory 
results in many cases, because in many cases, the portion of the distance spectrum which 
corresponds to Hamming weights above N(l — R) + 1 has a negligible effect on the overall 
bound. In j2], Agrell introduces a method (called C rule) in order to determine whether a 
codeword c is a zero-neighbor. 
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C rule: A codeword is a 0-neighbor if and only if it covers 3 no other nonzero codeword. 

In |3] , Ashikmin and Barg used this rule to derive explicit formulas for the weight 
spectrums of zero-neighbors for various codes. This includes the families of Hamming codes 
and second-order Reed-Muller codes. 

In order to upper bound the block error probability using the bounding technique in- 
troduced in this paper, we split the subcode C ex into two subcodes, C' ex and C" x , where C' ex 
contains all the codewords of C ox with Hamming weight IgWC {1, 2, N(l — R) + 1}, and 

contains the all-zero codeword and all the other codewords. The following upper bound 
holds: 

P.(C) = P c |o(C cx ) < P e |o(0 + ^.|o(0 (56) 

were P C \ (C' CX ) and P o | (C" x ) are the conditional block error probabilities of the subcodes C' ex 
and Cg X , respectively, given that the all-zero codeword was transmitted. We can upper bound 
P e |o(C" x ) by the union bound or the TSB, and we upper bound P e \o(C' ex ) by the MSFB (|2K|) . 
The partitioning of the subcode C ex into two subcodes C' ex and C" x is done following the 
adaptive algorithm introduced in Section El 



5 Applications 

This section demonstrates some numerical results of the improved upper bounds on the ML 
decoding error probability of linear block codes. We apply the bounds introduced in Sec- 
tions H3 and E] to various ensembles of parallel and serially concatenated codes. Throughout 
this section, it is assumed that the encoded bits are BPSK modulated, transmitted over an 
AWGN channel, and coherently detected. The effect of an expurgation of the distance spec- 
trum on the tightness of some upper bounds on the decoding error probability is exemplified 
as well. 

For the binary-input additive white Gaussian noise (BIAWGN) channel with BPSK mod- 
ulation, the conditional probability density function (pdf) for a single letter input is: 



p(y\0) = 

p(y|i) = 




where E s designates the energy of the symbol, and is the two-sided spectral power density 
of the channel. In order to calculate the SFB on P e | (C), we first calculate the terms A(p) 
and B(p), as defined in (|2T)l and (J27J), respectively. Clearly, for a continuous-output channel, 

3 A binary codeword ci is said to cover another codeword, C2, if C2 has zeros in all the positions where Ci 
has a zero. 
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the sums in (jzoj) and (j27j) are replaced by integrals. 
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where E denotes the statistical expectation, and X ~ N(0, 1). We also obtain that 

. ip- 1 
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A(p) +J B(p) = 2exp(-^)E 
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i+p 
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(58) 



(59) 



(60) 



Plugging (p)%j) - (p)Uj) into ()25|) . and (|50|l and minimizing over the interval < p < 1 will give 
us the desired bounds for P e | (C) and P\>\o{C'), respectively. 



5.1 Ensemble of Serially Concatenated Codes 

The scheme in Fig. El depicts the encoder of an ensemble of serially concatenated codes 
where the outer code is a (127, 99, 29) Reed-Solomon (RS) code, and the inner code is 
chosen uniformly at random from the ensemble of (8, 7) binary linear block codes. Thus, 
the inner code extends every symbol of 7 bits from the Galois field GF(2 7 ) to a sequence 
of 8 bits. The decoding is assumed to be performed in two stages: the inner (8, 7) binary 
linear block code is soft-decision ML decoded, and then a hard decision ML decoding is used 
for the outer (129, 99, 29) RS code. Due to the hard-decision ML decoding of the (127, 99, 
29) RS code, its decoder can correct up to t = L^f^J = 14 erroneous symbols. Hence, an 
upper bound on the average block error probability of the considered serially concatenated 
ensemble is given by 

127 

p ^ E OfW-p.) 13 " (61) 

i=t+l ^ ' 

where p s is the average symbol error probability of the inner code under soft-decision ML 
decoding. The symbol error probability p s of the inner code is either upper bounded by 
the ubiquitous union bound or the TSB, and this upper bound is substituted in the RHS 
of (|61|) . Since the rate of the inner code is rather high (it is equal to | bits per channel 
use), an expurgation of the distance spectrum seems to be attractive in order to tighten the 
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upper bound on the overall performance of the concatenated ensemble. Ashikmin and Barg 
[Sj show that the average expurgated distance spectrum of the ensemble of random linear 
block codes of length N and dimension K is given by 



We rely on the expurgated distance spectrum in ()62j) in order to get a tighter version of the 
union bound or the TSB on the symbol error probability p s of the inner code (where N = 8 
and K = 7). The expurgated union bound in Fig. 0] provides a gain of 0.1 dB over the union 
bound or TSB at block error probability of 10~ 4 , and the improvement in the tightness of 
the bound due to the distance spectrum expurgation is especially prominent at low values of 
SNR. Clearly, we take 1 as the trivial bound on p s (as otherwise, for low values of SNR, the 
union bound on p s may exceed 1, which gives in turn a useless upper bound on the decoding 
error probability of the ensemble). 

5.2 Turbo- Hamming Codes 

Let us consider an ensemble of uniformly interleaved parallel concatenated turbo-Hamming 
codes. The encoder consists of two identical (2 m — l,2 m — m — 1) Hamming codes as 
component codes, and a uniform interleaver operating on the 2 m — m — 1 information bits. 
The comparison here refers to the case where m = 10, so the two component codes are (1023, 
1013) Hamming codes, and the overall rate of the ensemble is R = 2m+m-i = 0-9806 bits per 
channel use. The value of the energy per bit to one-sided spectral noise density (^) which 
corresponds to this coding tare is 5.34 dB, assuming that communication takes place over 
a binary-input AWGN channel. In order to obtain performance bounds for the ensemble of 
uniformly interleaved turbo-Hamming codes, we rely on an algorithm for the calculation of 
the average input-output weight enumerator function (IOWEF) of this ensemble, as provided 
in ^3 Section 5.2]. As noted in \Fj\, the average distance spectrum of this ensemble is very 
close to the binomial distribution for a rather large range of Hamming weights (see Fig. [21(a)). 
Hence, one can expect that the upper bound introduced in Theorem ^ provides a tight 
bounding technique on the average block error probability of this ensemble. For this coding 
scheme, we note that regarding P e , there is no substantial improvement in the tightness of 
the overall upper bound if we upper bound P e \ (C") by the TSB instead of the simple union 
bound (see Fig. EJ). Among the bounds introduced in Section EJ the upper bound which 
combines the TSB and the MSFB is the tightest bound, especially for the low SNR range 
(see Fig- EI) ; referring to the bound in Theorem ^ the partitioning of codes in the considered 
ensemble relies on Algorithm 1 (see Section EJ). In Fig. [7J we provide a comparison between 
various upper bound on the bit error probability of this turbo-like ensemble. The tightest 
bound for the bit error analysis is the one provided in Theorem EJ combining the simplified 
DS2 bound with the union bound. It is shown in Fig. that the simplified DS2 provides 
gains of 0.16 dB and 0.05 dB over the MSFB at bit error probabilities of 10 _1 and 10 -2 , 
respectively. The simplified DS2 also provides gain of 0.08 dB over the TSB at bit error 
probability of 10 _1 . Unfortunately, a trivial expurgation of the average distance spectrum of 
uniformly interleaved turbo codes with two identical (2 m — 1, 2 m — m — 1) Hamming codes 





otherwise. 



(62) 



23 



as components (i.e., by nullifying the average distance spectrum at Hamming weights above 
2m + 1) has no impact on tightening the performance bounds of this ensemble. 



5.3 Multiple Turbo- Hamming Codes 

Multiple turbo codes are known to yield better performance, and hence, it is interesting 
to apply the new bounding techniques in Section El to these ensembles. The encoder of a 
multiple turbo-Hamming code is depicted in Fig. |H1 

Consider the ensemble of uniformly and independently interleaved multiple-turbo codes, 
where the components codes are identical systematic binary linear block codes of length N. 
Let S Wj hi denote the number of codewords of the i th component code with weight of the 
systematic bits equal to w and the weight of the parity bits equal to hi. The average number 
of codewords of the ensemble of multiple-turbo codes, with systematic-bits weight of w and 
overall weight I is given by 



From (Jfill)) and the algorithm to calculate the input-output weight enumerators of Hamming 
codes (see jT3 Appendix A]), it is possible to verify that the average distance spectrum of 
the ensemble of multiple turbo-Hamming codes with two independent uniform interleavers 
is very close to the binomial distribution for a relatively large range of Hamming weights 
(similarly to the plot in Fig. Efa)). Hence, the improved bounds provided in Section El are 
expected to yield good upper bounds on the decoding error probability. The comparison here 
refers to the case of m = 10, so the three component codes are (1023, 1013) Hamming codes. 
The overall rate of the ensemble is „ 2 jr~ m ~ 1 , = 0.9712 bits per channel use, and the channel 

2 m +2m— 1 1 ' 

capacity for this coding rate corresponds to = 5 dB. All the improved bounds that are 
evaluated here, incorporate the union bound as an upper bound on P e (C") (or Ph(C") for 
bit error probabilities). The numerical results of various upper bounds are shown in Fig. El 
for the block and bit error probabilities. As expected, the improvements that were obtained 
by the improved bounds (Theorems are more pronounced here than for the ensemble 
of turbo-Hamming code. For example, at bit error rate of 10 _1 , the simplified DS2 bound 
yields a gain of 0.12 dB over the TSB. A modest improvement of 0.05 dB was obtained at 
bit error rate of 10~ 2 . 

5.4 Random Turbo-Block Codes with Systematic Binary Linear 
Block Codes as Components 

Finally, we evaluate improved upper bound for the ensemble of uniformly interleaved par- 
allel concatenated (turbo) codes, having two identical component codes chosen uniformly 
at random and independently from the ensemble of systematic binary linear block codes. 
We assume that the parameters of the overall code are (N, K) , so the parameters of its 
component codes are ( N t K , K). In addition, the length of the uniform interleaver is K. 



A 



hi,h 2 ,h 3 s.t. 
w + hi + h 2 + h 3 





(63) 



24 



According to the analysis in j22] , the input-output weight enumeration of the considered 
ensemble is given by 



S(W,Z) = S ^S W ^W W Z 3 



W.J 



K 



1 + 2 { w 



w=l 



2 -(N-K) 



\ (N — K \ s h 

i \J2 i i^+2-— 



3=0 



N-K 

2 / N-K -* 

2 



E 

3=0 



where S w j denotes the number of codewords whose information sub-words have Ham- 
ming weight of w and the parity sub- word has Hamming weight j. We apply the im- 
proved bounds introduced in Section |3] to this ensemble where the parameters are set to 
(N,K) = (1144, 1000) (hence, the rate of the parallel concatenated ensemble is R = 0.8741 
bits per channel use). The plots of various upper bounds on the block and bit error prob- 
abilities are shown in Fig. The improved bounds yield the best reported upper bound 
on the block and bit error probabilities. For the block error probability, the upper bound 
which combines the MSFB with the union bound is the tightest bound; it achieve a gain of 
0.1 dB over the TSB, referring to a block error probability of 10~ 4 . A similar gain of 0.11 dB 
is obtained for the bit error probability, referring to a BER of 10~ 4 , referring to the bound 
which combined the union bound with the simplified DS2 bound (see Theorem 0J). 



6 Conclusions 

We derive in this paper tightened versions of the Shulman and Feder bound. The new 
bounds apply to the bit and block error probabilities of binary linear block codes under ML 
decoding. The effectiveness of these bounds is exemplified for various ensembles of turbo-like 
codes over the AWGN channel. An expurgation of the distance spectrum of binary linear 
block codes further tightens in some cases the resulting upper bounds. 
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Figures 




Figure 2: Plots of ^ and ^ as a function of the normalized Hamming weight (^), on 
a logarithmic scale. The plots refer to ensembles of random turbo-block codes with two 
identical systematic binary linear block codes as components; (a) A plot of ^ with N = 1000 
and R = 0.72 bits/Symbol, referring to the analysis of the block error probability, (b) A plot 

A' 

of -g 1 with N = 100 and R = 0.72 bits/Symbol, referring to the analysis of the bit error 
probability. 
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Figure 3: A scheme for an ensemble of serially concatenated codes where the outer code is 
a (127, 99, 29) Reed-Solomon (RS) code, and the inner code is chosen uniformly at random 
from the ensemble of (8,7) binary linear block codes. 




Figure 4: Various upper bounds on the block error probability of the ensemble of serially 
concatenated codes depicted in Fig. El The compared bounds are the tangential-sphere 
bound (TSB) and the union bound with and without expurgation of the distance spectrum; 
this expurgation refers to the ensemble of inner codes, chosen uniformly at random from the 
ensemble of (8,7) binary linear block codes. 
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Figure 5: A comparison between the upper bound which combines the UB with the SFB 
bound in it original form (Eq. (JJJ)) and the upper bound which combines the UB with the 
MSFB bound in ([25)1 . The comparison refers to the ensemble of uniformly interleaved turbo- 
Hamming codes where the two component codes are (1023, 1013) Hamming codes. The 
overall rate of the code is 0.973 bits per channel use. 
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Figure 6: Comparison between various upper bounds on the ML decoding block error prob- 
ability where the comparison refers to the ensemble of uniformly interleaved turbo-Hamming 
codes whose two component codes are (1023, 1013) Hamming codes. The compared bounds 
are the union bound (UB), the tangential-sphere bound (TSB), and two instances of the 
improved upper bound from Theorem ^ the UB+MSFB combines the MSFB with the 
union bound, and the TSB+MSFB is the upper bound which combines the MSFB with the 
tangential-sphere bound. 
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Figure 7: Comparison between various upper bounds on the ML decoding bit error prob- 
ability of the ensemble of (1033,1013) uniformly interleaved turbo-Hamming code. The 
compared bounds are the union bound (UB), the tangential- sphere bound (TSB), the upper 
bound from Theorem El which combines the union bound with the MSFB (UB+MSFB), and 
the upper bound from Theorem 0] which combines the union bound with the simplified DS2 
bound (UB+simplified DS2). 
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Figure 8: A multiple turbo-Hamming encoder. The encoder consists of parallel concatenated 
Hamming codes with two uniform, statistically independent interleavers. The code length is 
2 m + 2m — 1 and the code rate is R = ^^^n-i kits per channel use. 
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Figure 9: Comparison between various upper bounds on the ML decoding error probability, 
referring to the ensemble of uniformly interleaved multiple turbo-Hamming codes where the 
three component codes are (1023, 1013) Hamming codes (see Fig. EJ). The upper plot refers 
to upper bounds on the block error probability, and the compared bounds are the union 
bound (UB), the tangential-sphere bound (TSB), and the upper bound of Theorem^ which 
combines the union bound with the MSFB (UB+modified SFB). The lower plot refers to 
upper bounds on the bit error probability, and the compared bounds are the union bound 
(UB), the tangential-sphere bound (TSB), the upper bound of Theorem El which combines 
the union bound with the MSFB, and the upper bound of Theorem 0] which combines the 
union bound with the simplified DS2 bound (UB+ simplified DS2). 
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Figure 10: Comparison between upper bounds on the block and bit error probabilities for 
an ensemble of uniformly interleaved turbo codes whose two component codes are chosen 
uniformly at random from the ensemble of (1072, 1000) binary systematic linear block codes; 
its overall code rate is 0.8741 bits per channel use. The compared bounds under ML decoding 
are the tangential- sphere bound (TSB), and the bounds in Theorems [T] and |U The upper 
and lower plots provide upper bounds on the block and bit error probabilities, respectively. 
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